Logical Validation, Answer Merging and Witness Selection - A Study in Multi-Stream Question Answering

The paper presents an approach to multi-stream question answering (QA) using deep semantic parsing and logical validation for filtering answer candidates. A robust entailment check is accomplished by embedding the prover in a relaxation loop. Fallback strategies ensure a graceful degradation of performance in the case of parsing problems. The logical validity score is complemented by false-positive tests and heuristic quality indicators which also affect the selection of the most trusted answers. Separate criteria are used for choosing a suitable ‘witness’, i.e. a text passage which substantiates the answer. We present two experiments in which the method is applied for merging the results of various state-of-the-art QA systems. The evaluation demonstrates that the approach is applicable to heterogeneous QA streams – in particular it improves results for a combination of precision-oriented and recall-oriented answer streams. The method automatically adapts to these characteristics of a QA system by learning parameters from a training sample.